Skip to content

fix(decopilot): stream user prompt to other thread viewers#4461

Merged
pedrofrxncx merged 2 commits into
mainfrom
fix/stream-user-message-to-viewers
Jul 11, 2026
Merged

fix(decopilot): stream user prompt to other thread viewers#4461
pedrofrxncx merged 2 commits into
mainfrom
fix/stream-user-message-to-viewers

Conversation

@pedrofrxncx

@pedrofrxncx pedrofrxncx commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator

Problem

We stream the LLM output server-side to multiple consumers (several users viewing the same thread). But the user prompt is never put on the stream — it's only written to the DB. So a second viewer of a shared thread doesn't see the other user's message until a DB refetch happens (reconnect or a thread.status event), which feels laggy on a live thread.

Fix

Publish the materialized request message onto the same per-thread NATS JetStream subject as a transient data-user-message control chunk, at POST time, before the run's assistant chunks (so it sorts first).

Key point — large prompts are handled for free: the publish goes through the existing publishRawChunk → serializeChunk path, which already fragments anything over MAX_PUBLISH_BYTES (768 KiB) and drops over MAX_CHUNKED_BYTES (32 MiB). File attachments are already offloaded to object-storage URLs by uploadFileParts, so the payload is essentially text. No new large-payload machinery — this reuses the harness stream codec.

Server

  • user-message-stream.ts (new) — buildUserMessageChunk / isUserMessageControlChunk / publishUserMessage, mirroring run-status-stage.ts. Publish is best-effort (never fails the POST).
  • routes.ts — publish right after emitRequestMessage, in the same shouldPersistRequestMessage gate.
  • project-chunks.ts — filter the chunk out of the durable assistant projection. fenceFilter already drops it (it's fence-less), but the explicit type filter matches the data-run-status precedent and future-proofs against it ever being published with a run fence.

Client

  • thread-connection.ts — intercept data-user-message in handleChunk before the AI-SDK fold (the reassembler hardcodes role:"assistant"), and route it into the existing applyLocalMessagemergeAndSort/upsertById path. Dedup is free: the author's optimistic row shares the same id, and a later DB refetch reconciles created_at via preserveCreatedAt.

Testing

  • New unit tests for the helpers (user-message-stream.test.ts).
  • Extended project-chunks.test.ts to assert the chunk never leaks into projected assistant parts while normal assistant text still folds.
  • bun test (affected), tsc --noEmit (0 errors), bun run lint (0 errors) all pass.

Not covered / follow-up

Live two-viewer end-to-end wasn't driven headlessly here — worth a manual check (or an e2e in packages/e2e) with two sessions on one thread to confirm the prompt renders live for the non-author.


Summary by cubic

Mirror the user's prompt onto the per-thread run stream so other viewers see it instantly, fixing lag in shared threads. Uses a transient data-user-message control chunk and keeps the durable projection unchanged.

  • Bug Fixes
    • Server: publish a best-effort data-user-message chunk at POST time (before assistant chunks) via existing publishRawChunk fragmentation; filter it out of the assistant projection; gated with the same shouldPersistRequestMessage check.
    • Client: intercept data-user-message before the assistant fold and upsert via applyLocalMessage so the author's optimistic message dedupes and other viewers see it live.

Written for commit 4f24dff. Summary will update on new commits.

Review in cubic

The per-thread run stream carried only assistant/harness chunks, so a
second viewer of a shared thread never saw the other user's prompt until
a DB refetch. Publish the materialized request message onto the same
JetStream subject as a transient `data-user-message` control chunk at
POST time (before the run's assistant chunks), reusing publishRawChunk →
serializeChunk which already fragments payloads over MAX_PUBLISH_BYTES
and drops over MAX_CHUNKED_BYTES.

The client intercepts the chunk before the assistant fold (the AI SDK
reassembler hardcodes role:"assistant") and upserts it by id via
applyLocalMessage, so the author's optimistic copy dedupes while other
viewers get it live. The durable projection filters the chunk out; the
DB copy remains the emitRequestMessage write.
@pedrofrxncx pedrofrxncx merged commit 02a55b3 into main Jul 11, 2026
15 checks passed
@pedrofrxncx pedrofrxncx deleted the fix/stream-user-message-to-viewers branch July 11, 2026 04:31
decocms Bot pushed a commit that referenced this pull request Jul 11, 2026
PR: #4461 fix(decopilot): stream user prompt to other thread viewers
Bump type: patch

- decocms (apps/mesh/package.json): 4.11.3 -> 4.11.4

Deploy-Scope: both
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant